[XENAPI] Make server uri easily configurable via ENV or CLI opts.

Add a vm-stat command that just polls a vm for instantaneous statistics.

Signed-off-by: Alastair Tse <atse@xensource.com>

Fix 32-bit build.

Update hgignore list

minios : netfront driver fixes.

- Handle returned backend==NULL || mac==NULL, this leads sometimes to a crash.
- Remove unnecessary (and bogus) initialisation of np->rx.req_prod_pvt

Signed-off-by: Dietmar Hahn <dietmar.hahn@fujitsu-siemens.com>
Signed-off-by: Keir Fraser <keir@xensource.com>

xen: Remove unused elf32.c.
Signed-off-by: Keir Fraser <keir@xensource.com>

32-on-64: Fix an one-off error in compat_addr_ok().
Signed-off-by: Jan Beulich <jbeulich@novell.com>

Cleanups for unmodified (pv-on-hvm) driver building.

- some cleanup to mkbuildtree
- adjustment to the fake __supported_pte_mask definition (removing the
  export for x86-64 in native Linux is being discussed, so this shouldn't be
  i386 specific)
- remove odd/unused defines (mostly CONFIG_*) in overrides.mk

Signed-off-by: Jan Beulich <jbeulich@novell.com>

Replace mlock() calls with lock_pages().
Signed-off-by: Russell Blaine <russell.blaine@sun.com>

32-on-64: New set_address_size domctl for switching to compat mode.
From: Gerd Hoffmann <kraxel@suse.de>
Signed-off-by: Keir Fraser <keir@xensource.com>

32-on-64: Fix register-argument rearrangement to avoid clobbering the
fifth hypercall argument.
Signed-off-by: Keir Fraser <keir@xensource.com>

[XEND] Add missing ConsoleController.py

Signed-off-by: Alastair Tse <atse@xensource.com>

Support transparant gunzipping in the readnotes utility.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/xcutils/readnotes.c |   14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

libxc domain builder rewrite, linux builder

use new domain builder for the linux (aka generic elf) loader.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/Makefile              |    7 +-
 tools/libxc/xc_dom_compat_linux.c |  124 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 129 insertions(+), 2 deletions(-)

libxc domain builder rewrite, core bits.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/Makefile           |   14
 tools/libxc/xc_dom.h           |  261 +++++++++++++
 tools/libxc/xc_dom_binloader.c |  294 +++++++++++++++
 tools/libxc/xc_dom_boot.c      |  515 +++++++++++++++++++++++++++
 tools/libxc/xc_dom_core.c      |  773 +++++++++++++++++++++++++++++++++++++++++
 tools/libxc/xc_dom_elfloader.c |  283 +++++++++++++++
 tools/libxc/xc_dom_ia64.c      |  118 ++++++
 tools/libxc/xc_dom_powerpc64.c |  100 +++++
 tools/libxc/xc_dom_x86.c       |  559 +++++++++++++++++++++++++++++
 9 files changed, 2917 insertions(+)

libxc header fixups.

Make some arch-specific #defines for page table handling
available unconditionally, add a suffix to avoid name clashes.

The versions without suffix are defined depending on the
architecture like they used to, so code using them continues
to work.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/xg_private.h |  114 +++++++++++++++++++++++++++--------------------
 1 file changed, 67 insertions(+), 47 deletions(-)

Add more xc_error_code values.

XC_INVALID_PARAM
  such as asking for features unsupported by either xen or guest kernel.
XC_OUT_OF_MEMORY
  no comment ;)

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/xc_private.c |    4 ++++
 tools/libxc/xenctrl.h    |    2 ++
 2 files changed, 6 insertions(+)

Generate headers with arch-specific structs.

This patch adds a script to generate headers with arch-specific
structs which can be included on any architecture.  Can be used
to deal with structs of "foreign" architectures, needed for
32-on-64 support for example.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 Makefile                                  |   11 +-
 tools/Rules.mk                            |    2
 xen/Makefile                              |    4
 xen/include/public/foreign/Makefile       |   37 +++++++
 xen/include/public/foreign/mkchecker.py   |   58 +++++++++++
 xen/include/public/foreign/mkheader.py    |  153 ++++++++++++++++++++++++++++++
 xen/include/public/foreign/reference.size |   17 +++
 xen/include/public/foreign/structs.py     |   52 ++++++++++
 8 files changed, 331 insertions(+), 3 deletions(-)

libelf: use for readnotes utility.

This patch makes the readnotes utility use libelf.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/xcutils/readnotes.c |  277 +++++++---------------------------------------
 1 file changed, 45 insertions(+), 232 deletions(-)

libelf: use for hvm builder.

This patch switches over the hvm domain builder to libelf
(for loading hvmloader).

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/xc_hvm_build.c |  220 +++++++++++++--------------------------------
 1 file changed, 65 insertions(+), 155 deletions(-)

libelf: add to libxc

This patch makes libelf available to the tools, by symlinking the source
files and compiling them into libxc.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/Makefile |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

libelf: use for x86 dom0 builder.

This patch switches the x86 dom0 builder over to libelf.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 xen/arch/ia64/xen/domain.c  |   79 +++++------
 xen/arch/x86/domain_build.c |  303 +++++++++++++++-----------------------------
 xen/common/Makefile         |    4
 3 files changed, 148 insertions(+), 238 deletions(-)

add libelf: an ELF binary parser library.

This patch adds a library with a small collection of helper functions
to parse and load elf binaries.  The library handles endianess and
elfsize at runtime.

The patch also shuffles around the include files a bit.  Now there is
*one* include file holding all the elf structures
(xen/include/public/elfstructs.h) which is included by everyone who
needs them.

It's dead code with this patch only, putting the code into use happens
in followup patches.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>
---
 tools/libxc/xc_elf.h                |  525 -----------------------------------
 xen/arch/x86/boot/mkelf32.c         |    2
 xen/common/Makefile                 |    2
 xen/common/libelf/Makefile          |    4
 xen/common/libelf/README            |    1
 xen/common/libelf/libelf-dominfo.c  |  420 ++++++++++++++++++++++++++++
 xen/common/libelf/libelf-loader.c   |  156 ++++++++++
 xen/common/libelf/libelf-private.h  |   51 +++
 xen/common/libelf/libelf-relocate.c |  345 +++++++++++++++++++++++
 xen/common/libelf/libelf-tools.c    |  225 +++++++++++++++
 xen/include/public/elfstructs.h     |  527 ++++++++++++++++++++++++++++++++++++
 xen/include/public/libelf.h         |  238 ++++++++++++++++
 xen/include/xen/elf.h               |  490 ---------------------------------
 13 files changed, 1972 insertions(+), 1014 deletions(-)

[XEND] Preliminary console support in Xen API

Made serial/vnc consoles a 'fake' device so that we can take advantage
of storing state information in xenstore.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Fix HVM guest creation via Xen API

Fixed some of the type declarations for HVM devices, add better
handling for type mismatch errors.

Signed-off-by: Alastair Tse <atse@xensource.com>

Fix perfc=y build.
Signed-off-by: Keir Fraser <keir@xensource.com>

hvm: Define a global I/O access bitmap, allowing direct access to port 0x80.
Signed-off-by: Keir Fraser <keir@xensource.com>

[XEND] Fix bad patch merge for XendMonitor

Signed-off-by: Alastair Tse <atse@xensource.com>

netfront: Better fix for netfront_tx_slot_available().
Signed-off-by: Keir Fraser <keir@xensource.com>

emulator: Emulate IN/OUT/INS/OUTS. Fix CLTS.
Signed-off-by: Keir Fraser <keir@xensource.com>

emulator: Start filling in gaps for privileged instructions.
Signed-off-by: Keir Fraser <keir@xensource.com>

[XEND] Ignore socket.shutdown() exceptions

Add further comments about why that shutdown() exists.

Signed-off-by: Alastair Tse <atse@xensource.com>

32-on-64: Small fixes.
Signed-off-by: Jan Beulich <jbeulich@novell.com>

emulator: Add new hook functions for load/store privileged state.
Signed-off-by: Keir Fraser <keir@xensource.com>

During suspend, return immediately with a failure if the domain will
never suspend, instead of pointlessly retrying.

Signed-off-by: John Levon <john.levon@sun.com>

[HVM] Wake up APs immediately during restore.
From: Edwin Zhai <edwin.zhai@intel.com>
Signed-off-by: Keir Fraser <keir@xensource.com>

netfront: Check availability of grant references in start_xmit().
Signed-off-by: Tomonari Horikoshi <t.horikoshi@jp.fujitsu.com>

[XEND] Allow TCP XMLRPC address:port to be configured, if enabled.

Signed-off-by: Alastair Tse <atse@xensource.com>

Have the sxp parsing cope when the cpus field is a list, which can happen
after a reboot.

From Daniele Palumbo <daniele@retaggio.net>.

Signed-off-by: Ewan Mellor <ewan@xensource.com>

[XEND] Remove usage of set() in XendMonitor as it is a py2.4+ feature

Signed-off-by: Alastair Tse <atse@xensource.com>

[HVM][SVM] Add save/restore support.
From: Mats Petersson <Mats.Petersson@amd.com>
Signed-off-by: Keir Fraser <keir@xensource.com>

bimodal xenfb daemon: Fixes and cleanups.
From: Gerd Hoffmann <kraxel@suse.de>
Signed-off-by: Keir Fraser <keir@xensource.com>

32-on-64: All argument registers must be zero extended to 64 bits
(%rdx was missed).
Signed-off-by: Keir Fraser <keir@xensource.com>

[XEND] Fix typo in XendNode

Signed-off-by: Alastair Tse <atse@xensource.com>

Fix device reordering that occurs when the config gets read into dictionaries.

Signed-off-by: Ewan Mellor <ewan@xensource.com>

Make domctl/sysctl interfaces 32-/64-bit invariant.
This kills off a fair amount of unpleasant CONFIG_COMPAT shimming and
avoids needing to keep the compat paths in sync as these interfaces
continue to develop.
Signed-off-by: Keir Fraser <keir@xensource.com>

[XEND] Add debug class to Xen API plus non-standard debug.wait().

debug.wait(seconds) will just do a time.sleep() so the task progress
support can be tested.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Move decorate() to XendAPI.__new__ and have a static flag to
ensure it only runs at instantiation time, and only once per Xend
instance. Otherwise, decorate() runs on every invocation of
/usr/sbin/xend, even if it is xend stop.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XENAPI] Make test script attempt to login with blank credentials
before prompting for passwords.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Cleanup old domains in statistics monitor.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XENAPI] Add vif-list to xapi.py script

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Output exception to log files if XendAPI fails to instantiate.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XENAPI] Update test scripts to move from VBD.image to creating VDI's
with the uri attribute.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Add support for multiple storage repositories.

Splits XendStorageRepository into a helper class and
XendQCowStorageRepo as the QCoW File backed repository. This gives us
the opportunity to introduce a pseudo storage repository to represent
the old way of specifying block devices. This is the
XendLocalStorageRepo.

Note that this still introduces a non-Xen API 'uri' attribute to VDIs
in order to support querying VDIs for their actual location relative
to the Xen host.

The QCoW backed repository is now named QCoW, and the pseudo-SR is
named Local.

This removes the 'image' attribute backwards-compat hack in VBDs in
the Xen API.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Add missing Xen API methods

* Added session.get_all, task.get_by_name_label,
  host.get_by_name_label, VM.add_to_otherConfig,
  VM.remove_from_otherConfig

* Added implementations for VM.get_VCPUs_policy,
  VM.get_platform_std_vga, VM.get_vm_by_uuid,
  VM.get_platform_localtime, VM.get_platform_clock_offset,
  VM.get_platform_enable_audio, VM.get_platform_keymap,
  VM.get_otherConfig, VM.set_actions_after_shutdown,
  VM.set_actions_after_reboot, VM.set_actions_after_suspend,
  VM.set_actions_after_crash, VM.set_platform_std_VGA,
  VM.set_platform_serial, VM.set_platform_keymap,
  VM.set_platform_localtime, VM.set_platform_clock_offset,
  VM.set_platform_enable_audio, VM.set_otherConfig, VBD.destroy,
  VBD.get_io_read_kbs, VBD.get_io_write_kbs, VBD.get_all, VIF.destroy,
  VIF.get_VM, VIF.get_name, VIF.get_MTU, VIF.get_MAC, VIF.get_type,
  VIF.get_device, VIF.get_io_read_kbs, VIF.get_io_write_kbs,
  VIF.get_all, VTPM.destroy, VTPM.get_all

* Save devid of a device on creation in XendDomainInfo

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Make sure UUID in state store are not stored as unicode
objects.

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Prevent invalid arguments for destroy event channels.

Signed-off-by: Alastair Tse <atse@xensource.com>

Test scripts for Xen API updated to support Async support

[XEND] Add Task support in Xen API implementation.

Added progress tracking to some common methods like VM.start so the
progress during async invocation.

Signed-off-by: Alastair Tse <atse@xensource.com>

[linux] remove gratuitous differences with plain linux.

Signed-off-by: Christian Limpach <Christian.Limpach@xensource.com>

[XEND] Add missing XendMonitor.py

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Add simple VCPUs, VBDs, VIFs statistics monitoring

Keep track of xentop like statistics to expose through the Xen API

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Strip suffix from device name and add support for 'VBD.type'

Signed-off-by: Alastair Tse <atse@xensource.com>

[linux] Disable GENERIC_TIME until we have a xen clocksource.

Signed-off-by: Christian Limpach <Christian.Limpach@xensource.com>

Fix crash if some secondary CPUs cannot be initialised.
Signed-off-by: Keir Fraser <keir@xensource.com>

[QEMU] Clear TD status field explicitly when it's fetched.

In current Qemu-dm, UHC will set some status bits of TD in status
updating stage, but never process the status bit if relevant condition
does not occur, leaving it as it is. When a TD is fetched with some
status bits are set to 1, it will return to Guest OS with these bits
set to 1 even this TD is executed successfully. Some Windows OS,
e.g. Windows 2000, will check status bits of TD in UHC interrupt
routine, treat it as a unsuccessful one if some status bits are set to
1 and discard the data. Other Windows OS just check USBSTS of UHC,
ignoring status field of TD unless the value of USBSTS indicates
occurrence of error.

With this patch, USB mouse/tablet in Windows 2000 works correctly.

Signed-off-by: Xinmei Huang <xinmei.huang@intel.com>

bimodal blkback: Support multiple ring protocols.

This is needed for 32-on-64 support.  Right now there are three
protocols: native, x86_32 and x86_64.  If needed it can be extended.

Interface changes (io/blkif.h)
 * Define the x86_32 and x86_64 structs additionally to the native
   version.
 * Add helper functions to convert them requests to native.

Backend changes:
 * Look at the "protocol" name of the frontend and switch ring
   handling accordingly.  If the protocol node isn't present it
   assumes native protocol.
 * As the request struct is copied anyway before being processed (for
   security reasons) it is converted to native at that point so most
   backend code doesn't need to know what the frontend speaks.
 * In case of blktap this is completely transparent to userspace, the
   kernel/userspace ring is always native no matter what the frontend
   speaks.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>

Fix error message in network-bridge script.
From: Luke Crawford <lsc@prgmr.com>
Signed-off-by: Keir Fraser <keir@xensource.com>

libxenguest: Fix xc_resume() build for non-x86.
Signed-off-by: Keir Fraser <keir@xensource.com>

Add BSD-style license to top of hvm_op.h public header file.
Signed-off-by: Keir Fraser <keir@xensource.com>

Close save-after-restore race.

Make xc_linux_save() wait for the frame_list_list MFN to be updated by the
domain before trying to use it. Make Linux set the top-level MFN /after/
updating the other MFN lists.

Signed-off-by: John Levon <john.levon@sun.com>

[LIBXC] Provide weak stub default implementations of xc_hvm_save() and
xc_hvm_restore() for architectures with no full implementation.
Signed-off-by: Keir Fraser <keir@xensource.com>

linux: Make resend_irq_on_evtchn() non-static for ia64 use.
Signed-off-by: Keir Fraser <keir@xensource.com>

[HVM] fix save/restore on 64b HV

Enable hvm_ctxt ops for compat mode to fix HVM save/restore on 64b HV.

Signed-off-by: Zhai Edwin <edwin.zhai@intel.com>

x86: flush_tlb_mask() should take into account specified VA to flush
on local processor.
Signed-off-by: K. Y. Srinivasan <ksrinivasan@novell.com>

bimodal pvfb backend: Deal with both 32 and 64 bit frontends.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>

bimodal blkfront: Create a new node "protocol" in xenstore, add the
protocol name it speaks there.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>

bimodal: pvfb frontend

Create a new node "protocol" in xenstore, add the protocol name it
speaks there.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>

bimodal: header file with protocol names.

Signed-off-by: Gerd Hoffmann <kraxel@suse.de>

linux: Fix enable_irq() crash by removing a BUG_ON() assumption in our
event-channel retrigger() function. Also clean up bitmap usages.
Signed-off-by: Keir Fraser <keir@xensource.com>

[XM-TEST] Allow building of ramdisk on x86_64

Signed-off-by: Alastair Tse <atse@xensource.com>

[LINUX] Force .data segment to be 4k aligned on i386
by backporting git changeset c06cb8b1c4d25e5b4d7a2d7c2462619de1e0dbc4

This is required for kexec/kdump.

Signed-off-by: Ian Campbell <ian.campbell@xensource.com>

[XEND] Save model attribute for HVM VIFs

This solves the problem of HVM VIFs forgetting their model config on
reboot.

http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=867

Signed-off-by: Alastair Tse <atse@xensource.com>

[XEND] Fix get_dev_property_by_uuid

This patch replaces calls to get_dev_property() by calls to
get_dev_property_by_uuid() in XenAPI.py and fixes the implementation
of get_dev_property_by_uuid.

I am adding a test case to the xapi grouptests to verify the fixes.
There's a FIXME note in the test case which should be looked at.

Signed-off-by: Stefan Berger <stefanb@us.ibm.com>

[PV-on-HVM] Make PV drivers on HVM kernels work on older kernels after
update to 2.6.18.

Signed-off-by: Ian Campbell <ian.campbell@xensource.com>

Fix failure to localise xm by preventing cd from writing to stdout
which gets incorrectly captured as a value for the bash for loop.
Signed-off-by: Keir Fraser <keir@xensource.com>

[PVFB] Set keyboard layout of PVFB from configuration file.

For example:
keymap = "ja"

Default remains en-us.

Signed-off-by: Takanori Kasai <kasai.takanori@jp.fujitsu.com>
Signed-off-by: Tomohiro Takahashi <takatom@tokyo.ssg.fujitsu.com>
Signed-off-by: Masami Watanabe <masami.watanabe@jp.fujitsu.com>

[PVFB] Support keymap localisation.

Conversion to the scan code uses QEMU's function.
Moreover, keyboard support for PVFB uses QEMU's keymaps.
Keyboard default layout is en-us.

Signed-off-by: Junko Ichino <ichino.junko@jp.fujitsu.com>
Signed-off-by: Takanori Kasai <kasai.takanori@jp.fujitsu.com>
Signed-off-by: Hirofumi Tsujimura <tsujimura.hirof@jp.fujitsu.com>
Signed-off-by: Tsunehisa Doi <Doi.Tsunehisa@jp.fujitsu.com>

VTPM_TOOLS: Added support for QEMU to communicate with vTPM over UNIX
socket for HVM guests.

Signed-off-by: Vinnie Scarlata <vincent.r.scarlata@intel.com>

Put gdb version in a variable in gdbbuild script.
Signed-off-by: Xin Li <xin.b.li@intel.com>

linux/i386: widen physical address range

32on64 supports 168Gb for 32-bit guests, hence at least 38 bits must
be specified here. The foreign frame bit is MFN bit 31, so the maximum
valid value here would be 43.

Signed-off-by: Jan Beulich <jbeulich@novell.com>

merge

linux/i386: relax highpte pinning/write-protecting
Signed-off-by: Jan Beulich <jbeulich@novell.com>

[XEN] Consolidate multiple defintions of ELFNOTE_* accessor macros

Signed-off-by: Ian Campbell <ian.campbell@xensource.com>

[LINUX] Fix native build.

Signed-off-by: Ian Campbell <ian.campbell@xensource.com>

[XEN] Avoid void* arithmetic in kexec ELF note manipulations.

From: Christoph Egger <Christoph.Egger@amd.com>
Signed-off-by: Ian Campbell <ian.campbell@xensource.com>

Implement clean return from save/restore failure (so that original
domain can continue execution).
Signed-off-by: Andrei Petrov <andrei.petrov@xensource.com>

[SVM] Remove ASID logic. Errata prevent this feature being used
reliably in current SVM processor implementations.
Signed-off-by: Keir Fraser <keir@xensource.com>

Re-enable CONFIG_HOTPLUG_CPU by deafult for -xen? kernels.

Signed-off-by: Steven Hand <steven@xensource.com>

[PYGRUB] Plumb bootloader I/O through xenconsole.

 - xend forwards console traffic between the running bootloader and a pty
   which it writes to the store so the xenconsole client can see it.
 - the xenconsole client handles the domain's console pty changing.
 - xm create no longer runs the bootloader.
 - pygrub gets '-i' option to explicitly request an interactive session.
 - xend unlocks the domain list during bootloading so that "xm console"
   can see the domain and attach to its console pty.

Signed-off-by: Tim Deegan <Tim.Deegan@xensource.com>

[PYGRUB] Add python module for POSIX ptsname(2) function.
Signed-off-by: Tim Deegan <Tim.Deegan@xensource.com>